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Abstract 



Analytic treatment of a non-equilibrium random system with large degrees 
of freedoms is one of most important problems of physics. However, little 
research has been done on this problem as far as we know. In this paper, 
we propose a new mean field theory that can treat a general class of a non- 
equilibrium random system. We apply the present theory to an analysis for an 
associative memory with oscillatory elements, which is a well-known typical 
random system with large degrees of freedoms. 

PACS numbers: 87.10.+e, 05.90.+m, 89.70,+c 



Typeset using REVTgX 



1 



The analytic treatment of a non-equilibrium random system with large degrees of free- 
doms is one of most important problems of physics. However, little research has been done 
on this problem as far as we know. In this paper, we propose a new mean field theory that 
can treat a general class of a non-equilibrium random system. We apply the present theory 
to an analysis for an associative memory with oscillatory elements, which is a well-known 
typical random system with large degrees of freedoms. 

Historically, there are two important studies regarding the non-equilibrium systems with 
large degrees of freedoms that cannot be treated by conventional equilibrium statistical 
mechanics, since we can not define the Lyapunov function with "bottoms". Kuramoto || 
theoretically analyzed the mutual entrainment of uniformly coupled oscillators with scat- 
tered natural frequencies (Kuramoto theory). His model corresponds to a mean field model 
of a ferromagnet in equilibrium statistical mechanics. Kuramoto utilized the spilite of sta- 
tistical mechanics, namely, a notion of macroscopic order-parameters, to investigate his 
non-equilibrium system with large degrees of freedoms. Daido || numerically analyzed the 
quasi-entrainment of randomly coupled oscillators with scattered natural frequencies. His 
model corresponds to the Sherrington-Kirkpatrick model of a spin glass |2j in equilibrium 
statistical mechanics. A mean field theory should be developed for non-equilibrium random 
systems with frustration that follows the history of mean field theories in the equilibrium 
statistical mechanics. However, this kind of non-equilibrium random system has not yet 
been theoretically analyzed (see for the random system without frustration). This is a 
famous "open problem" in physics 0. That is reason why we have proposed the present 
theory. 

On the other hand, the Lyapunov function can not be defined in spin-like systems with 
the non-monotonic reaction function either. In the self-consistent signal-to-noise analysis 
(SCSNA) [|TIJ, the notion of a macroscopic order parameteris also introduced to analyze 
these frustrated spin-like systems, which cannot be treated by conventional equilibrium 
statistical mechanics. Note that the results of applying the SCSNA to simple random spin 
systems |10,|Il|, i.e., the Sherrington-Kirkpatrick model and the Hopfield model [l], coincide 



with those of the replica theories [121,131 



In this paper, we propose a new theoretical framework for an oscillator associative mem- 
ory model with scattered natural frequencies in memory retrieval states. This system can 
be considered as a typical example of non-equilibrium random systems with large degrees 
of freedoms. The present theory makes a bridge between the SCSNA and the Kuramoto 
theory. Using the same procedure, we can easily treat a glass oscillator system |5[]. Our 
theory is reduced to the Kuramoto theory in the finite loading case. When all oscillators 
have uniform natural frequencies, our theory coincides with the previously proposed theories 
UH in the equilibrium statistical mechanics for an XY spin system. 

The mutual entrainment is an important notion of a non-equilibrium system with large 
degrees of freedoms. In uniformly coupled oscillators, there is a unique stable state, the 
ferromagnetic phase in the phase space. On the other hand, in frustrated systems, there are 
many stable states in the phase space. We need to elucidate the properties of the mutual 
entrainment in each stable phase (ferromagnetic phase and glass phase). Our new theory 
describes a phenomenon of the mutual entrainment in the ferromagnetic phase (memory 
retrieval). Thus, we numerically study a degree of the mutual entrainment in the glass 
phase (spurious memory retrieval). It is numerically shown in this paper that almost all 
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oscillators synchronize under memory retrieval, but desynchronize under spurious memory 
retrieval when setting optimal parameters. Thus, it is possible to determine whether the 
recalling process is successful or not using information about the synchrony/ asynchrony. 

In general, when the coupling is sufficiently weak, the high- dimensional dynamics of a 
coupled oscillator system can be reduced to the phase equation Let us consider the 

following simplified model, 

d(f)i N 

= uJi + J^JijSin^j - + (1) 
at j=1 

where N is the total number of oscillators, 0« is the phase of the %— th oscillator, and uoi is 
the natural frequency assumed to be randomly distributed over the whole population with 
a density denoted by the symmetric distribution g(u), i.e., g(u) = g(—u). Note that the 
average of Ui may be set to zero without loss of generality. The theory presented below can 
be easily extended to treat the system with other any distribution g(u). Jij and denote 
an amplitude of a synaptic weight and a synaptic delay, respectively. In order to investigate 
the nature of frustrated non-equilibrium systems, we have selected the following generalized 
Hebb learning rule to determine Jy and fy, 

C ij = J ij exp(i(3 ij ) = ±j2^, £f = exp(^), (2) 

iV n=i 

as their typical example. {0f }i=i,...,jv,/i=i,-,p are the phase patterns to be stored in the 
network and are assigned to random numbers with a uniform probability in [0, 2iv]. Here, 
we define a parameter a (loading rate) such that a = p/N. In the equilibrium limit of this 
model, that is, g(u) = S(u), the storage capacity given by a c = 0.038 ||. 

We put Si = exp(i0j) for the sake of simplicity. The order parameter m' 1 , which measures 
the overlap between system state s, and embedded pattern is defined as 

1 N - 

JV 3=1 

We obtain order parameter equations of the present system by applying the following ma- 
nipulations. 

• First, assuming a self-consistent local field for each of oscillators, a distribution of Sj 
under g(uJi) is formally derived by the Kuramoto theory. 

• Second, we estimate the contribution of randomness, that is, the uncondensed patterns 
in the present case by the SCSNA, and determine the local field in a self-consistent 
manner. 

• Finally, the order parameter equations are obtained using the self-consistent local field. 

A detailed derivation of the present theory will be discussed elsewhere. Here, we assume 
m 1 = 0(1) and m M = 0(l/y/N) for \i > 1 (uncondensed patters). Then, the following two 
dimensional equations for the order parameters are obtained, 



3 



m=((X{x 1 ,x 2 ,Z)))^, (4) 
U={{F 1 {x 1 ,x 2 ,Z))) n ^, (5) 

where ((• • •)) Xi X2 , is taken to mean the Gaussian average over xi, x 2 and condensed pattern 
((' ' x 2 £ = (I I Dx 1 Dx 2 ■ ■ -)^. The pattern superscripts 1 of m are omitted for brevity. 
The self-consistent mean field h, X, F\ and the Gaussian measure Dx\Dx 2 are expressed as 
follows, 



P 2 = 2 ^ _ jjy , h = £m + x 1 +ix 2 , (7) 

x 2 , £)=hj dxg (|ft|a?) Vl - x 2 , (8) 
Fi(xi,x 2 ,0 = 

y 1 da; ^ + (|/L|ar)^ Vl - x 2 . (9) 

Here, Z7 corresponds to the susceptibility, which measures the sensitivity to external fields. 
A distribution of resultant frequencies ZJ in the memory retrieval state, which is denoted as 
p(uJ), becomes 



9 H 1 + ^ 

p(G7) = rS(ZJ) + I D Xl Dx 2 V 7 , (10) 

1 + ^ 



|/i| 2 

r = y DxiDx 2 |/i| y dxg , (11) 

where r measures the ratio between the number of synchronous oscillators and the system 
size N. We now consider the relationships between the present theory and the previously 
proposed theories. For the equilibrium limit, g(x) = 5(x), we obtain 

x=b F ^ = ^ pfa) = m, (12) 

\h\ 2\h\ 

which coincide with the replica theory and the SCSNA ||. On the other hand, regarding 
the uniform-system limit, a = 0, our theory reproduces the Kuramoto theory as 



m 



m / dxg (\m\x) Vl — x 2 , (13) 



9[^ + ¥ 



p(uJ) = rd(uJ) + v i = 2 7 ~. (14) 



|m| 2 

r = |m| / dx^ (|m|x) . (15) 
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As mentioned before, our theory has made a bridge between the equilibrium-frustrated 
system and the non-equilibrium-uniform system. In addition, we have presented a systematic 
way of analytical treatments for the non-equilibrium random systems. If CV, is assigned to 
random numbers with Gaussian in complex plane, Re[CV,] ~ J\f(l/N,a/2N), Im[CV,] ~ 
A/"(0, a/2N), the order parameter equation consists with Eq. ^ under constraint U — 0. The 
detail results will be presented elsewhere. We can also treat Daido's glass oscillator system 
(with real number interaction) || using the same procedure. 

In the following analyses, we choose a system with g{uj) = (2-na 2 )^ 1 / 2 exp(— uj 2 /2a 2 ). 
Fig. [TJ(a) shows a phase diagram in the (|m|, a, a) space, which was obtained by numerically 
solving these order parameter equations. A cross-section of this curved surface at a = 
coincides with the results of the SCSNA || and the Replica theory ||. Furthermore, a 
cross-section of this curved surface at a = is equal to a result of the Kuramoto theory 
||. Thus, our theory bridges the gap between these theories. Fig. |l|(b) shows the values 
of critical memory capacity a c for various values of a. a c decreases monotonically as a 
increases. The critical value of o at a c = is given by a c = 0.62, which coincides with that 
of the SK theory. Figures @(a), (b),(c), and (d) display \m\ values vs a for various values of 
a, where the solid curves are obtained theoretically, and the plots show results obtained by 
numerical simulation. According to these figures, the theory is in good agreement with the 
simulation results. 

Next, we examined the distributions of the resultant frequencies uJj over the whole popu- 
lation of oscillators in the memory retrieval state and the spurious memory state. Here, the 
resultant frequencies oJi were calculated by using the long time average of dfa/dt. The plots 
in Figures |3](a) and (b) show the values of resultant frequencies {cl'i}i=i,...,jv versus a, that 
is, a bifurcation diagram. Fig. |^(a) denotes uJj distributions in memory retrieval states, and 
Fig. ^|(b) represents those in spurious memory states. These results show that there exists 
a region a that satisfies the two conditions: all oscillators mutually synchronize in memory 
retrieval, and oscillators desynchronize in spurious memory retrieval. Thus, it is possible 
to determine whether the recalling process is successful or not only using the information 
about the synchrony /asynchrony, when proper a is given. In Figures |](a) and (b), all phase 
values 4>i continued drifting toward negative directions due to an offset of scattered u>i. In 
other samples, all 4>i continued drifting toward positive directions. 

We investigated the effect of the system size N on the cJj distribution. The solid curves in 
Fig. £| indicate the distributions of the resultant frequencies p(u) in Eq. [!(]. The histograms 
in Fig. [| show the results from the numerical simulation. As shown in Figures |](a)(a')(a"), 
the theory agrees well with the simulation results in memory retrieval states. As shown 
in Figures f|(b)(b')(b"), there exists a delta-peak that indicates the mutual entrainment of 
a large population of oscillators in memory retrieval states. However, in spurious memory 
states, the distribution of the average frequency is gentle compared to the distribution in 
memory retrieval states, which indicates the quasi-entrainment of oscillators || . According 
to these figures, we believe the phenomena mentioned before are invariant to the system 
size N. The reason why all oscillators mutually synchronize in memory retrieval states, 
but desynchronize in spurious states, is because effective interactions among oscillators in 
memory states (as in ferromagnetic states [|]]) are different from interactions in spurious 
memory states (as in spin glass states in which the system is strongly frustrated. 

The phase description proposed here can be considered as a minimum model of neural 
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networks based on oscillatory activities that is mathematically solvable. Chaos neural net- 
works yield rich phenomena as discussed here, but can not be easily analyzed, except with 
simulations. Since the present analysis corresponds to the replica symmetric approximation, 
we have noted that it should be extended to the replica symmetry breaking in order to 
properly treat the spurious states (spin glass states), and this remains for a future work. 

In the field of neuroscience, a growing number of researchers have been interested in the 
synchrony of oscillatory neural activities because physiological evidence of their existence 
has been obtained in the visual cortex of a cat fl4yi5| . Much experimental and theoretical 
research exists regarding the functional role of synchronization. One of the more interesting 
hypotheses is called synchronized population coding, which was proposed by Phillips and 
Singer. However, its validity is highly controversial [ 16| . In this paper, we numerically 
showed the possibility of determining if the recalling process is successful or not using in- 
formation about the synchrony/ asynchrony. If we consider information processing in brain 
systems, the solvable toy model presented in this paper may be a good candidate for showing 
the validity of a synchronized population coding in the brain, and we believe the present 
analysis may strongly influence debate on the functional role of synchrony. 

The authors thank Ms. M. Yamana, Mr. M. Yoshioka and Dr. M. Shiino for their 
valuable discussions. This work was partially supported by Grants-in-Aid for Scientific 
Research in Priority Area (2) No. 07252219, Grants-in-Aid for the Encouragement of Young 
Scientists No. 0782 and JSPS Research Fellowships for Young Scientists. 
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FIGURES 



FIG. 1. (a) Phase diagram in (\m\,a,a) space, (b) a vs critical memory capacity a c . 



FIG. 2. Values of \m\ vs a (solid curves theoretically obtained, and plots obtained by numerical 
simulation), (a) a = 0.0 (p = 1, N = 1000). (b) a = 0.01 (N = 2000). (c) a = 0.02 (N = 2000). 
(d) a = 0.03 (N = 2000). 

FIG. 3. Values of resultant frequencies {wi}^^...^ vs a, N = 2000 and a = 0.0315. (a) 
Memory pattern retrieval, (b) Spurious memory pattern retrieval. 



FIG. 4. Distribution of resultant frequencies state, a = 0.01, a = 0.32. Solid curves show- 
theoretical results of p(tJ) in Eq. 10, but delta-peaks at U = are not indicated. Figs. (b)(b')(b") 
display detailed distributions of Figs, (a) (a') (a") at UJ = 0, respectively. (a)(b) N = 2000. 20 
trials. (a')(b') N = 4000. 12 trials. (a")(b") N = 8000. 12 trials. 
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